Over the past decade, area-based CCD cameras have been successfully applied to a wide variety of industrial machine vision applications. Conventional area-based CCD cameras are based on a two dimensional matrix of photoelements. Typically, the entire matrix is scanned 30 times per second producing a complete frame of video data every 33 milliseconds.
Frame-based vision systems are the most common platforms for capturing and processing the video images produced by area-based cameras. In a typical area-based vision system, a full video frame from the area camera is digitized and captured into a frame buffer. Under microprocessor control, software algorithms are utilized to process the stored image and extract the features of interest. The pixel processing operations are usually accelerated by a hardware coprocessor. Based on the extracted features, external outputs may be triggered for process monitoring and/or control.
Area-based cameras and frame-based vision systems are popular for several reasons. Area-based cameras are easy to understand, set up and focus. Area-based cameras are inexpensive and widely available. Frame-based vision systems are plentiful and offer rich software toolsets at a reasonable price.
It is expected that frame-based vision systems will continue to dominate the machine vision industry for some time. However, frame-based cameras and vision systems have some serious limitations which restrict their use in many important industrial applications.
In spite of success to date, area-based vision systems have some significant limitations. Standard area-based CCD cameras are typically limited in resolution to approximately 800.times.600. High resolution area cameras are available but they are very expensive. The scanning frequency of most frame-based vision systems is constrained by the RS-170 video standard. This limits the speed at which images can be acquired and processed. Many systems are further complicated by interlaced image capture and a 4:3 pixel ratio. Further, it is often difficult to achieve uniform lighting over the two dimensional region that is scanned by the area camera. Non-uniform lighting reduces the overall performance of the vision system. Still further, frame buffers are finite and can only capture an image of finite length. Many applications require continuous processing of infinitely large images. Finally, frame-based vision systems with strobe lights are difficult to properly synchronize to continuous processes. It is difficult to achieve 100% inspection of a continuous process with a frame-based vision system.
These shortcomings restrict the extent to which area-based vision can be successfully applied. There is a broad class of vision problems for which the utilization of area cameras is either overly expensive, overly complex or simply impractical to utilize due to the fundamental limitations of area-based vision. For these applications, a different class of cameras and processors is required.
There is a large class of machine vision applications which can be considered "high performance". These applications are characterized by high resolution imaging, high speed image acquisition, high speed image processing, continuous image acquisition and processing.
Continuous web inspection, for example, is one of the most demanding industrial vision applications. A typical high speed web inspection application inspects a web which is 24 inches wide, with a web velocity of 400 inches per second (2000 feet/minute), performs 100% surface inspection and detects defects as small as 0.020 inches. A vision system that inspects such a web would require a cross-web resolution of at least 2048 pixels and would need to continuously scan and process over 70 million pixels per second.
A frame-based vision system is impractical for such a demanding application. Although there have been attempts to apply frame-based vision to these applications, the results are typically characterized by limited processing capability, high cost, complexity or all of the above.
Line scan cameras exhibit certain qualities which enable them to overcome the limitations of area-based cameras and be successfully applied to high performance vision applications. Line scan cameras are based on a linear CCD array with a single row of pixels. A continuous, two-dimensional image of a moving object is created by repeatedly scanning the single row of pixels.
Line scan cameras have the following advantages over area-based cameras. Linear array sensors are available with horizontal resolutions of over 6000 pixels. High resolution linear arrays are easier to fabricate and therefore less expensive than their area-based equivalents. By scanning the linear array at a high frequency, very high vertical image resolution is attained. Scanning of the linear array can be triggered continuously, thereby generating a continuous image of the moving object. Line scan cameras view only a single line on the object of interest. This narrow line is much easier to uniformly illuminate than the two dimensional region required for area-based cameras. Optical encoders synchronized to object motion may be utilized to trigger the horizontal line capture.
Line scan cameras are well suited to high performance vision applications because of their ability to generate high resolution, continuous images at high scanning frequencies. The major limitation of conventional line scan cameras is low light sensitivity at high scanning frequencies. When the horizontal scanning frequency is high, the time interval over which each photoelement is gathering light is proportionately reduced. For example, if a line scan camera is being scanned at 20,000 lines per second, each pixel is only integrating light for 50 microseconds before it is transferred to the output shift register. Thus, to operate at high scanning speeds, extremely high intensity lights are required. Such high intensity illumination is often impractical and, as such, conventional line scan cameras are typically limited to scanning at 5 kHz or less.
Time Delay and Integration (TDI) line scan cameras have emerged as a solution to high speed line scanning. TDI cameras retain the positive features of line scan cameras but have dramatically improved light sensitivity. A TDI image sensor is comprised of a number of adjacent linear rows of photoelements. The motion of the object of interest is precisely synchronized to the TDI sensor. As the object moves along the TDI sensor array, multiple exposures of the same portion of the object are accumulated down each column of photoelements. The total accumulated charge from the multiple exposures is then transferred to the output shift register in the same manner as a conventional linear array camera. The net effect is an increase in the exposure time for each pixel by a factor equal to the number of rows in the TDI sensor. For example, a TDI sensor with 96 stages exhibits up to 80 times the light sensitivity of the equivalent conventional line scan camera. With this vastly improved light sensitivity, scanning at high frequencies is achievable with reasonable light sources.
TDI cameras are not without their drawbacks, however. In order to obtain a sharp image, the TDI sensor must be precisely synchronized to the motion of the object being viewed. The instantaneous scan-to-scan velocity of the object must be tightly controlled. Furthermore, since the image is exposed multiple times, the object must be very nearly flat over the length that it is projected onto the TDI array.
With the greatly improved light sensitivity provided by TDI technology, high scanning frequencies are possible utilizing conventional light sources such as fluorescent tubes. High speed line scanning is no longer limited by light sensitivity. Rather, the problem now is processing the torrent of video data that is produced by the TDI camera. A 2048 element TDI sensor scanned at 35,000 Hz, continuously produces over 70 million pixels per second. To accommodate this data rate, multiple output taps are provided. For example, the DALSA CT-E1-2048 TDI camera provides eight taps, each responsible for scanning 256 contiguous pixels of the 2048 element array.
TDI line scan cameras are able to generate high resolution images of objects (or webs) moving at extremely high speeds. This capability opens up a world of new, high performance vision applications which were previously impossible. However, the number of industrial applications that utilize TDI cameras has been, to date, extremely limited because there does not exist a cost effective vision processor that is capable of performing sophisticated two dimensional image processing on the continuous deluge of data that issues from the TDI camera.
To illustrate the order of magnitude of processing power required for TDI applications, the web inspection example described earlier utilizing an 8-tap TDI camera with 2048 pixel horizontal resolution scanning at 35,000 lines per second will produce the required image. However, each output tap produces nearly 9 million pixels per second totalling over 70 million pixels per second for the camera. A modest set of vision algorithms will require approximately 25 operations per pixel resulting in a sustained demand for over 1.75 billion operations per second (1,750 MOPs). Clearly this is far beyond the capabilities of most frame-based vision systems, especially those that process pixel data utilizing software algorithms.
There are many excellent frame-based vision processors available today. However, the unique characteristics of high speed line scan cameras impose additional requirements on the vision processor. A high speed line scan vision processor must have no frame limitations, continuous processing capability, real-time pixel processing at line scan rates and processing of parallel taps.
Several approaches have been taken to accomplish the processing of high speed line scan camera data, with the most common approach of utilizing a conventional frame-based vision processor with a line scan camera interface. Unfortunately, the fundamental limitations of frame-based architectures (described above) renders them unsuitable for most high speed line scan processing applications. (see B. Harvey, "The Application of Flexible Machine Vision Architecture to the Inspection of Continuous Process Materials", SME Technical Paper #MS89-165, Society of Manufacturing Engineers, 1989.)
In order to achieve extremely high processing rates, a parallel processing architecture is required. Many parallel processing architectures have been proposed for high speed number crunching. (see Z. Hussain, "Digital Image Processing: Practical Applications of Parallel Processing Techniques", Ellis Horwood Limited, West Sussex, England, 1991.) However, the "pipeline" architecture is particularly well suited to machine vision. (see N. Storey and R. C. Staunton, "An Adaptive Pipeline Processor for Real-Time Image Processing", Proceedings of SPIE Volume 1197, pp 238-246, 1989 and J. W. V. Miller, "Architectural Considerations for Linear-Camera Vision Systems", Proceedings of SPIE Volume 1615, pp 312-319, 1991.)
There are a few commercial systems available which have been designed specifically for line scan processing. Typically, these systems were designed to process conventional line scan data and lack the power required to process TDI camera data in real-time (see J. W. V. Miller, "A survey of Image Processing Architectures for Linear Array Cameras", Conference Proceedings from Vision 90, pp 7-1-7-11, November 12-15, 1990.). A few systems exist which can perform realtime processing at high pixel data rates, but are characterized by high cost and excessive complexity. Furthermore, these systems are often custom built for specific applications and therefore lack flexibility.